Software  Engineering  Institute 


Driving  Out  Technical  Risk 
by  Blending  Architecture, 
Process,  and  Project 
Discipline 

Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

James  McHale,  Robert  Nord 

In  collaboration  with: 

Luis  Carballo  (Bursatec) 


Carnegie  Mellon 


©2012  Carnegie  Mellon  University 


Form  Approved 
OMB  No.  0704-0188 


Report  Documentation  Page 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

MAY  2012 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2012  to  00-00-2012 

4.  TITLE  AND  SUBTITLE 

Driving  Out  Technical  Risk  by  Blending  Architecture,  Process,  and 

Project  Discipline 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

Carnegie  Mellon  University, Software  Engineering 

Institute, Pittsburgh, PA, 15213 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

SEI  Architecture  Technology  User  Network  Conference  (SATURN  2012),  May  7-11, 2012,  St  Petersburg, 
FL. 

14.  ABSTRACT 

The  SEI  recently  worked  with  Bursatec,  the  IT  arm  of  the  Mexican  stock  exchange,  to  help  build  an 
enhanced  online  financial  trading  engine.  The  end  product  had  aggressive  goals  for  performance  and 
delivery,  and  it  had  to  function  flawlessly.  Factors  complicating  a  solution  included  scope  outside  the 
organizations  recent  experience,  combining  key  technologies  in  new  ways,  and  a  constant  stream  of  new 
requirements?challenges  familiar  to  many  in  industry.  To  manage  these  challenges  while  instilling 
disciplined  but  nimble  engineering  practices,  the  SEI  employed  its  Team  Software  Process  with 
architecture-centric  engineering  to  set  an  integrated  architecture/developer  team  in  motion.  The  blend  of 
technical,  process,  and  project-management  discipline  was  used  to  systematically  address  technical  risk. 
This  collaborative  approach  offers  help  to  organizations  to  set  an  architecture/developer  team  in  motion 
using  mature,  disciplined  engineering  practices  that  produce  quality  software  quickly. 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF  PAGES 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

28 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Copyright  2012  Carnegie  Mellon  University 

This  material  is  based  upon  work  funded  and  supported  by  the  Department  of  Defense  under  Contract 

No.  FA8721-05-C-0003  with  Carnegie  Mellon  University  for  the  operation  of  the  Software  Engineering  Institute, 

a  federally  funded  research  and  development  center. 


NO  WARRANTY 

THIS  CARNEGIE  MELLON  UNIVERSITY  AND  SOFTWARE  ENGINEERING  INSTITUTE  MATERIAL  IS  FURNISHED  ON 
AN  “AS-IS”  BASIS.  CARNEGIE  MELLON  UNIVERSITY  MAKES  NO  WARRANTIES  OF  ANY  KIND,  EITHER  EXPRESSED 
OR  IMPLIED,  AS  TO  ANY  MATTER  INCLUDING,  BUT  NOT  LIMITED  TO,  WARRANTY  OF  FITNESS  FOR  PURPOSE  OR 
MERCHANTABILITY,  EXCLUSIVITY,  OR  RESULTS  OBTAINED  FROM  USE  OF  THE  MATERIAL.  CARNEGIE  MELLON 
UNIVERSITY  DOES  NOT  MAKE  ANY  WARRANTY  OF  ANY  KIND  WITH  RESPECT  TO  FREEDOM  FROM  PATENT, 
TRADEMARK,  OR  COPYRIGHT  INFRINGEMENT. 


This  material  has  been  approved  for  public  release  and  unlimited  distribution  except  as  restricted  below. 

The  Government  of  the  United  States  has  a  royalty-free  government-purpose  license  to  use,  duplicate,  or  disclose  the  work, 
in  whole  or  in  part  and  in  any  manner,  and  to  have  or  permit  others  to  do  so,  for  government  purposes  pursuant  to  the 
copyright  license  under  the  clause  at  252.227-7013  and  252.227-7013  Alternate  I. 

Internal  use:*  Permission  to  reproduce  this  material  and  to  prepare  derivative  works  from  this  material  for  internal  use  is 
granted,  provided  the  copyright  and  “No  Warranty”  statements  are  included  with  all  reproductions  and  derivative  works. 

External  use:*  This  material  may  be  reproduced  in  its  entirety,  without  modification,  and  freely  distributed  in  written  or 
electronic  form  without  requesting  formal  permission.  Permission  is  required  for  any  other  external  and/or  commercial  use. 
Requests  for  permission  should  be  directed  to  the  Software  Engineering  Institute  at  permission@sei.cmu.edu. 

*  These  restrictions  do  not  apply  to  U.S.  government  entities. 


-  Software  Engineering  Institute 


Carnegie  Mellon 


©2012  Carnegie  Mellon  University 


A  New  Trading  Platform 


FINANCIAL  TIMES 

January  9, 2012  12:45  am 

Mexico’s  BMV  to  be  among 
fastest  bourses  by  May 


“Mr  Tellez  [BMV  president]  said  that  the  new  system’s  initial 
configuration  will  be  able  to  handle  200,000  transactions  per  second, 
and  take  less  than  1 00  microseconds  to  process  a  single  order. 

That  catapults  the  BMV  into  the  same  league,  in  terms  of  latency,  as 
large  exchanges  such  as  Nasdaq  OMX,  NYSE  Euronext,  Deutsche 
Borse  and  the  London  Stock  Exchange.  ” 
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Experience  and  Results 


Architecture  coaching,  coupled  with  the 
discipline  of  the  Team  Software  Process, 
built  a  competent  architecture  team  and  an 
architecture  with  successful  evaluation 
quickly  -  less  than  six  months. 

The  project  objectives  were  met. 

•  Schedule  -  finished  on  time. 

•  Quality  -  early  trials  and  quality  metrics  suggest  reliability  and  quality 
goals  were  met. 

•  No  known  defects  carried  into  final  cycle! 

•  Performance  -  a  day’s  worth  of  transactions  can  be  processed  in 
seconds. 
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The  Opportunity 


Background: 

•  Bolsa  Mexicana  de  Valores  (BMV) 
operates  the  Mexican  financial  markets. 

•  Bursatec  is  the  technology  arm  of  BMV. 

•  BMV  desired  a  new  trading  engine  to 
replace  the  existing  stock  market  engine 
and  integrate  the  options  and  futures 
markets. 

•  Bursatec  committed  to  deliver  a  trading 
engine  in  8-10  quarters. 

•  Bursatec  approached  the  SEI  for  support 
during  design  and  development  to  improve 
its  software  delivery  capability. 
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The  Project  -1 


Business  Goals: 

•  High  performance  (as  fast  or  faster  than  anything 
out  there) 

•  Reliable  and  of  high  quality  (the  market  cannot 
go  down) 


•  Scalable  (able  to  handle  spikes  and  long-term 
growth  in  trading  volume) 
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The  Project  -2 


Architecture  Decisions: 

•  Development  in  Java  (lower  Total  Cost  of  Ownership) 

•  Low  Latency  Communication  Multicast  Network 

•  In  memory  data  storage  during  trading  session 

•  Hot-Hot  High  Availability  configuration 

•  Parallel  processing  in  Java  Virtual  Machine  (JVM) 

•  Horizontal  scalability 

Functional  Requirements: 

•  Order  routing  with  FIX  protocol  interconnect  to  current 
legacy  systems. 

•  Combined  Cash  and  Derivatives  markets  with  a  single 
Control  Workstation. 

•  Separate  Market  Data  and  Index  calculation  system. 


Trading  Engine  Quality  and  Other  Attributes 


Quality  Attributes 

•  Under  1ms  processing  latency 

•  Horizontal  scalability 

•  Redundant  high  availability 
system 

•  Warm  dual  redundant  system 

•  Automatic  testing  framework 
(one  day  turnaround  attribute) 

•  Localize  business  rules  changes 
in  specific  modules 


Other  Attributes 

•  Backward  compatible  with  current 
systems 

•  Combined  platform  for  both 
markets 

•  Run  on  Commodity  hardware 

•  86  order  type/attribute 
combinations  (30  in  current 
system) 

•  Real  time  updates  to  status  of 
system  via  Control  Workstation. 
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System  Context  of  the  Trading  Engine 
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Complicating  Factors 


Given  the  context,  one  would  expect  risks  due  to: 

•  Large  project  -  scope  beyond  the  organization’s  recent  experience 

—  #  of  person-months 

—  #  KLOC/function  points 

—  #  of  interconnecting  platforms 

—  #  of  individual  projects 

•  Inexperience  -  available  staff  talented  but  young  and  key  implementation 
technologies  never  used  together  formally 

•  Constant  stream  of  new  requirements  and  changes  to  business  rules 
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Solution  Integrates  High-Value  Architecture  and 
Team  Practices 


Team  Software  Process 

•  Proven  technology 

•  Management  and  measurement 
across  the  project  lifecycle 

•  Building  high-performance 
teams 

•  Key  managers  familiar  with 
technology  through  word-of- 
mouth  and  literature. 


Architecture-Centric  Engineering 

•  Proven  technology 

•  Technical  aspect  of  the 
early  project  lifecycle  activities 

•  Architecting  to  meet  business 
objectives 

•  Key  managers  familiar  with 
technology  via  training  courses. 


Architecture  drove  the  work  breakdown  structure  (WBS)  and  provided 
a  robust  framework  for  requirements  management. 
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Architecture  Drives  the  Lifecycle 


Two  iterative  processes  based  on  the  architecture  of  the  system: 

Design  cycles  (1,  2)  Implementation  cycles  (3,  5,  6) 

The  goal  is  to  design  a  system  The  goal  is  to  implement  the 
that  ensures  business  success.  system  according  to  the  design. 
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ACE  /  TSP  Design,  Analysis,  and  Implementation 


Architecture  Trade 
Analysis  Method 
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Example  Design  and  Implementation  Strategy 
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Select  Process  Data 


Planned  % 
Actual  % 


Effort  distribution  by  phase  blocks 
(%  of  total  task  hours) 

-208  eKLOC  in  24  months 

Complete  functionality  of  previous 
system  and  new  functionality 

Latency  target  1  msec, 
achieved  0.1  msec 


Architectural  design  practices  were12%  of  the  total  cost  but  were  key  in 
meeting  the  technical  requirements  and  are  estimated  to  have  reduced 
the  implementation  costs  by  1 0%-1 5%  (due  to  avoided  functionality  and 
clean  design) 

Only  15%  effort  in  testing  -  compared  to  normally  equal  distribution 
between  coding  and  testing;  higher  than  usual  quality  achieved 
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Accomplishments 


Performance 

Latency  and  throughput  metrics  greatly  exceeded  initial  expectations 
(0.1  msec.  vs.  1.0  msec.) 

Quality 

Very  low  defect  count  in  validation  test.  Error  density  0.1  error  per  KLOC 
compared  to  “normal”  of  0.5-1 .0 

Defects  encountered  have  not  modified  the  architecture 

Testing  framework  allowed  a  smooth  continuous  integration 

Cost  &  Schedule 

•  Team  achieved  EVERY  Milestone  (internal  and  external)  on  time  and  budget 
(including  unplanned  new  functionality),  with  the  planned  number  of  people. 
No  “forced  march”. 
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Key  Takeaways 


Investment  in  early  architecture  and  team  practices  drive  the  lifecycle  and 
plays  a  role  is  managing  risk. 

Earlier  identification  and  resolution  of  defects  reduces  the  cost  of  rework. 

Iterative  and  incremental  approach  fosters  collaboration  and  facilitates 
handoffs  reducing  the  cost  of  delay. 

Architectural  practices  and  TSP  provide  a  disciplined  framework  for 
measuring  and  managing  structured  intellectual  activity  related  to  the 
product,  process,  and  project. 
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Questions? 


For  more  information:  Combining  Architecture-Centric  Engineering  with  the  Team 
Software  Process,  Technical  Report,  CMU/SEI-2010-TR-031,  December  2010 
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Contact  Information 


TSP  Initiative 

James  W.  Over 
TSP  Initiative  Lead 

iwo@sei.cmu.edu 


RTSS  Program 

Linda  Northrop 
RTSS  Program  Director 

lmn@sei.cmu.edu 


Jim  McHale 
TSP  Mentor  Coach 

idm@sei.cmu.edu 


Felix  Bachmann 
Architecture  Mentor  Coach 

fb@sei.cmu.edu 


Business  Development 

David  Scherb  dscherb@sei.cmu.edu 
Greg  Such  qsuch@sei.cmu.edu 


SEI  website  at  www.sei.cmu.edu  (~/tsp  or  -/architecture) 
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ADDITIONAL  INFORMATION 
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Project  History 


Cycle  1  -  Architecture 

•  Completed  Jan.  2010,  demonstrated  architecture  coaching,  evaluation  of  comm, 
packages,  built  test  framework 

Cycle  2  -  Infrastructure  implementation 

•  Completed  Apr.  2010,  included  successful  ATAM  (documentation  noticeably 
thorough,  no  significant  new  architectural  risks  discovered) 

Cycle  3  -  Basic  functions  and  main  performance  loop 

•  Completed  July  2010,  good  quality,  performance  exceeding  requirements  by  more 
than  a  factor  of  5 

Cycle  4  -  Non-TSP  cycle,  outside  evaluation  by  world-class  experts 

•  Completed  Aug.  2010,  JVM  &  high-speed  redundant  communications 

Cycle  5  -  Full  normal  operations,  complete  performance  loop 

•  Completed  Jan.  201 1 

Cycle  6  -  Full  functionality  incl.  startup,  shutdown,  &  maintenance  modes 

•  Completed  July  201 1  (additional  scope  extended  scheduled  June  finish) 
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Project  History  -2 


Cycle  7  -  System  Test  /  Integration  Test  w/  Legacy  Systems  (starting  Aug.  2011) 
Cycle  8  -  Acceptance  Test  /  Parallel  Test  (starting  Dec.  201 1 ) 

Cycle  9  -  User  Test  /  Deployment  (starting  Jan.  2012) 

•  Testing  activities  overlapped  in  part  due  to  the  Matching  Engine  readiness  being 
AHEAD  of  other  interfacing  systems 

•  Includes  internal  test  group,  internal  operations,  brokerage  firms  testing  (functional, 
HA,  throughput  ,and  DRP  tests) 

•  Operations  testing  detected  (as  of  Jan.  2012)  <  50  unique  defects  in  200+  KLOC 

Go-Live  Scheduled  May  2012  (NOW!) 


=-  Software  Engineering  Institute  CarnegieMellon  ©2010,  2011  Carnegie  Mellon  University 


Getting  Started 


TSP/ACE  is  introduced  into 
an  organization  on  a 
project-by-project  basis. 


TSP  Introduction  Steps 

1. 

Start  by  identifying  candidate  projects,  architects, 
and  internal  architecture  and  TSP  coach 
candidates. 

2. 

Train  senior  management. 

3. 

Train  the  selected  teams  and  their  managers, 
then  launch  the  project. 

4. 

Monitor  the  projects  and  make  adjustments  as 
needed. 

5. 

Expand  the  scope  to  include  additional  projects 
and  teams. 

6. 

Create  or  expand  the  pool  of  available  SEI- 
authorized  architects,  instructors  and  coaches. 

7. 

Repeat  starting  at  step  3. 
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Selecting  Pilot  Projects 


Pick  3  to  5  medium-to  large-sized  pilot  projects. 

•  8  to  15  team  members 

•  4  to  18  month  schedule 

•  Software-intensive  new  development  or  enhancement 

•  Representative  of  the  organization’s  work 

•  Important  projects 

Select  teams  with  members  and  managers  who  are  willing  to  participate. 

Consider  the  group  relationships. 

•  Contractors 

•  Organizational  boundaries 

•  Internal  conflicts 
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Architecture  Training 


Six  Courses 
Software  Architecture 
Principles  and  Practices* 

Documenting 
Software  Architectures 

Software  Architecture 
Design  and  Analysis 

Software  Product  Lines 
ATAM  Evaluator  Training 
ATAM  Leader  Training 
ATAM  Observation 


Certificate  Programs  Certification 


Software  ATAM 

Architecture  Evaluator 

Professional 


ATAM 

Leader 
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Software 
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required  to  receive 
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*:  available  through  e-learning 
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PSPSM  anc|  jsp  Training 


Personal  Software  Process 
(PSPSM)  training  is  essential  to 
successful  TSP  implementation. 

•  TSP  Executive  Seminar  (1  day  for  top-level  execs,  middle  managers) 

•  TSP  Team  Leader  Training  (3  days  for  team  leads,  affected  managers) 

•  PSP  Fundamentals  (5  days  for  software  developers) 

•  TSP  Team  Member  Training  (3  days  for  other  disciplines) 
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Intellectual  Property 


Personal  Software  Process,  PSP,  Team  Software  Process,  and  TSP  are 
service  marks  of  Carnegie  Mellon  University. 

Architecture  Tradeoff  Analysis  Method  and  ATAM  are  registered  in  the 
U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon  University. 


Software  Engineering  Institute  CamegieMellon 


Driving  Out  Technical  Risk  by  Blending 
Architecture,  Process,  &  Project  Discipline 

©2012  Carnegie  Mellon  University 


NO  WARRANTY 


THIS  CARNEGIE  MELLON  UNIVERSITY  AND  SOFTWARE  ENGINEERING  INSTITUTE 
MATERIAL  IS  FURNISHED  ON  AN  “AS-IS"  BASIS.  CARNEGIE  MELLON 
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